feat: add development automation tools - #175

Please address the comments from this code review:

## Overall Comments
- Consolidate duplicate branch naming and size validation logic between the Python script and the GitHub Actions workflow to avoid maintaining two sources of truth.
- Defaulting git diff stats to HEAD~1 may not capture all PR changes in multi-commit PRs—consider using the PR base branch (origin/${BASE_BRANCH}) by default for more accurate metrics.
- The PR scope checker script is becoming quite monolithic; consider refactoring static configuration (e.g., acceptable file-type combinations) into a separate module or config file for easier maintenance.

## Individual Comments

### Comment 1
<location> `scripts/check_pr_scope.py:44-57` </location>
<code_context>
+    stdout, stderr, code = run_command(["git", "diff", "--shortstat", range_spec])
+    lines_changed = 0
+    if code == 0 and stdout:
+        # Parse shortstat format like "1 file changed, 10 insertions(+), 2 deletions(-)"
+        shortstat = stdout.strip()
+        if shortstat:
+            parts = shortstat.split(",")
+            for part in parts:
+                part = part.strip()
+                if "insertions" in part or "deletions" in part:
+                    nums = "".join(c for c in part if c.isdigit())
+                    if nums:
+                        lines_changed += int(nums)
+
</code_context>

<issue_to_address>
**suggestion (bug_risk):** Shortstat parsing may overcount lines if both insertions and deletions are present.

The logic currently sums all digits in parts containing 'insertions' or 'deletions', which may lead to double-counting. Using regex to extract and sum insertion and deletion counts separately would improve accuracy.

```suggestion
    import re
    # Get lines changed using shortstat
    stdout, stderr, code = run_command(["git", "diff", "--shortstat", range_spec])
    lines_changed = 0
    if code == 0 and stdout:
        # Parse shortstat format like "1 file changed, 10 insertions(+), 2 deletions(-)"
        shortstat = stdout.strip()
        if shortstat:
            insertions = 0
            deletions = 0
            ins_match = re.search(r"(\d+)\s+insertions?\(\+\)", shortstat)
            del_match = re.search(r"(\d+)\s+deletions?\(-\)", shortstat)
            if ins_match:
                insertions = int(ins_match.group(1))
            if del_match:
                deletions = int(del_match.group(1))
            lines_changed = insertions + deletions
```
</issue_to_address>

### Comment 2
<location> `scripts/check_pr_scope.py:252-257` </location>
<code_context>
+    if files:
+        file_types = set()
+
+        for file in files:
+            if file.endswith((".py", ".js", ".ts", ".java", ".cpp", ".c", ".h")):
+                file_types.add("code")
</code_context>

<issue_to_address>
**suggestion (bug_risk):** File type detection logic may misclassify files based on substring matches.

Substring checks like 'test' or 'config' may match unintended files (e.g., 'contest.py'). Use more precise methods, such as directory prefixes or exact extensions, to avoid misclassification.

```suggestion
            elif (
                file.startswith("tests/")
                or file.endswith(("_test.py", "_test.js", "_test.ts", "_test.java", "_test.cpp", "_test.c", "_test.h"))
            ):
                file_types.add("tests")
            elif (
                file.startswith("config/")
                or file.endswith((".yml", ".yaml", ".json", ".toml", ".cfg"))
                or file in ("config.py", "config.js", "config.ts", "config.java", "config.cpp", "config.c", "config.h")
            ):
                file_types.add("config")
```
</issue_to_address>

### Comment 3
<location> `Makefile:24` </location>
<code_context>
+	ruff check --diff src/ tests/ scripts/
+	ruff format --check src/ tests/ scripts/
+	pylint --rcfile=pyproject.toml src/ tests/ scripts/
+	mypy src/ scripts/training/ scripts/database/
+	bandit -r src/ -c pyproject.toml
+	bandit -r tests/ -c pyproject.toml -s B101
</code_context>

<issue_to_address>
**suggestion:** Mypy invocation may fail if directories do not exist.

Add checks to ensure these directories exist before running mypy, or use wildcards to prevent errors if they're missing.

```suggestion
	mypy src/**/*.py scripts/training/**/*.py scripts/database/**/*.py
```
</issue_to_address>

### Comment 4
<location> `scripts/check_pr_scope.py:23` </location>
<code_context>
    result = subprocess.run(cmd, capture_output=True, text=True, check=False)
</code_context>

<issue_to_address>
**security (python.lang.security.audit.dangerous-subprocess-use-audit):** Detected subprocess function 'run' without a static string. If this data can be controlled by a malicious actor, it may be an instance of command injection. Audit the use of this call to ensure it is not controllable by an external resource. You may consider using 'shlex.escape()'.

*Source: opengrep*
</issue_to_address>

### Comment 5
<location> `scripts/check_pr_scope.py:216` </location>
<code_context>
    base_for_stats = args.base if args.base else f"{args.branch}~1"

</code_context>

<issue_to_address>
**suggestion (code-quality):** Replace if-expression with `or` ([`or-if-exp-identity`](https://docs.sourcery.ai/Reference/Rules-and-In-Line-Suggestions/Python/Default-Rules/or-if-exp-identity))

```suggestion
    base_for_stats = args.base or f"{args.branch}~1"
```

<br/><details><summary>Explanation</summary>Here we find ourselves setting a value if it evaluates to `True`, and otherwise
using a default.

The 'After' case is a bit easier to read and avoids the duplication of
`input_currency`.

It works because the left-hand side is evaluated first. If it evaluates to
true then `currency` will be set to this and the right-hand side will not be
evaluated. If it evaluates to false the right-hand side will be evaluated and
`currency` will be set to `DEFAULT_CURRENCY`.
</details>
</issue_to_address>

### Comment 6
<location> `scripts/check_pr_scope.py:217` </location>
<code_context>
    head_for_stats = args.head if args.head else args.branch

</code_context>

<issue_to_address>
**suggestion (code-quality):** Replace if-expression with `or` ([`or-if-exp-identity`](https://docs.sourcery.ai/Reference/Rules-and-In-Line-Suggestions/Python/Default-Rules/or-if-exp-identity))

```suggestion
    head_for_stats = args.head or args.branch
```

<br/><details><summary>Explanation</summary>Here we find ourselves setting a value if it evaluates to `True`, and otherwise
using a default.

The 'After' case is a bit easier to read and avoids the duplication of
`input_currency`.

It works because the left-hand side is evaluated first. If it evaluates to
true then `currency` will be set to this and the right-hand side will not be
evaluated. If it evaluates to false the right-hand side will be evaluated and
`currency` will be set to `DEFAULT_CURRENCY`.
</details>
</issue_to_address>

### Comment 7
<location> `scripts/check_pr_scope.py:49-57` </location>
<code_context>
def get_git_stats(
    base: str = "HEAD~1",
    head: str = "HEAD",
) -> Tuple[int, int, List[str]]:
    """Get git statistics for a commit range."""
    # Use git range operator base...head for both name-only and stats
    range_spec = f"{base}...{head}"

    # Get number of files changed
    stdout, stderr, code = run_command(["git", "diff", "--name-only", range_spec])
    if code != 0:
        print(f"❌ Git diff error: {stderr}")
        return 0, 0, []

    files = stdout.split("\n") if stdout else []
    num_files = len([f for f in files if f.strip()])

    # Get lines changed using shortstat
    stdout, stderr, code = run_command(["git", "diff", "--shortstat", range_spec])
    lines_changed = 0
    if code == 0 and stdout:
        # Parse shortstat format like "1 file changed, 10 insertions(+), 2 deletions(-)"
        shortstat = stdout.strip()
        if shortstat:
            parts = shortstat.split(",")
            for part in parts:
                part = part.strip()
                if "insertions" in part or "deletions" in part:
                    nums = "".join(c for c in part if c.isdigit())
                    if nums:
                        lines_changed += int(nums)

    return num_files, lines_changed, files

</code_context>

<issue_to_address>
**issue (code-quality):** Use named expression to simplify assignment and conditional [×2] ([`use-named-expression`](https://docs.sourcery.ai/Reference/Default-Rules/refactorings/use-named-expression/))
</issue_to_address>

### Comment 8
<location> `scripts/check_pr_scope.py:173` </location>
<code_context>
def main():
    """Main function."""
    import argparse

    parser = argparse.ArgumentParser(description="Check PR scope compliance")
    parser.add_argument(
        "--branch",
        default="HEAD",
        help="Branch to check (default: HEAD)",
    )
    parser.add_argument(
        "--base",
        help="Base branch/commit for range comparison (e.g., main, origin/main)",
    )
    parser.add_argument(
        "--head",
        help="Head branch/commit for range comparison (default: current branch)",
    )
    parser.add_argument(
        "--strict",
        action="store_true",
        help="Strict mode - fail on any warning",
    )
    args = parser.parse_args()

    print("🔍 Checking PR Scope Compliance")
    print("=" * 50)

    all_passed = True

    # Check branch name
    print("📋 Checking branch name...")
    if not check_branch_name_quality():
        all_passed = False

    # Check commit message
    print("\n📝 Checking commit message...")
    if not check_commit_message_quality(args.base, args.head):
        all_passed = False

    # Check file and line limits
    print("\n📊 Checking size limits...")
    # Determine base for git stats
    base_for_stats = args.base if args.base else f"{args.branch}~1"
    head_for_stats = args.head if args.head else args.branch
    num_files, lines_changed, files = get_git_stats(base_for_stats, head_for_stats)

    print(f"   Files changed: {num_files}")
    print(f"   Lines changed: {lines_changed}")

    if num_files > 50:
        print(f"❌ Too many files changed! Max 50 allowed, got {num_files}")
        if args.strict:
            all_passed = False

    if lines_changed > 1500:
        print(f"❌ Too many lines changed! Max 1500 allowed, got {lines_changed}")
        if args.strict:
            all_passed = False

    # List changed files
    if files:
        print("\n📁 Files changed:")
        for file in files[:10]:  # Show first 10
            if file.strip():
                print(f"   - {file}")
        if len(files) > 10:
            print(f"   ... and {len(files) - 10} more")

    # Check for mixed concerns (improved logic to reduce false positives)
    print("\n🎯 Checking for mixed concerns...")
    if files:
        file_types = set()

        for file in files:
            if file.endswith((".py", ".js", ".ts", ".java", ".cpp", ".c", ".h")):
                file_types.add("code")
            elif file.endswith((".md", ".rst", ".txt", ".adoc")):
                file_types.add("docs")
            elif "test" in file.lower() or file.startswith("tests/"):
                file_types.add("tests")
            elif "config" in file.lower() or file.endswith(
                (".yml", ".yaml", ".json", ".toml", ".cfg"),
            ):
                file_types.add("config")
            elif "docker" in file.lower() or "Dockerfile" in file:
                file_types.add("docker")
            elif "api" in file.lower() or "endpoint" in file.lower():
                file_types.add("api")
            elif (
                "model" in file.lower() or "ml" in file.lower() or "ai" in file.lower()
            ):
                file_types.add("ml")

        # Allow reasonable combinations that are commonly acceptable
        acceptable_combinations = [
            {"code", "tests"},  # Feature + tests
            {"code", "tests", "config"},  # Feature + tests + config
            {"code", "tests", "docs"},  # Feature + tests + docs
            {"code", "tests", "config", "docs"},  # Feature + tests + config + docs
            {"code", "config"},  # Code changes with config
            {"code", "docs"},  # Code changes with docs
            {"config", "docs"},  # Config changes with docs
            {"tests", "config"},  # Tests with config
        ]

        # Flag as mixed concerns only if we have more than 4 types OR
        # if we have an unusual combination that's not in acceptable list
        is_mixed_concerns = len(file_types) > 4 or (
            len(file_types) > 2 and file_types not in acceptable_combinations
        )

        if is_mixed_concerns:
            print(f"⚠️  Mixed concerns detected: {', '.join(sorted(file_types))}")
            print("   Consider splitting into separate PRs")
            print(
                "   Acceptable combinations include: code+tests, code+tests+config, etc.",
            )
            if args.strict:
                all_passed = False
        elif len(file_types) > 2:
            # Show info about acceptable combinations but don't fail
            print(f"ℹ️  Multiple file types detected: {', '.join(sorted(file_types))}")
            print("   This combination is acceptable for a single PR")

    print("\n" + "=" * 50)
    if all_passed:
        print("✅ PR Scope Check PASSED")
        return 0
    print("❌ PR Scope Check FAILED")
    print("\n💡 Remember the rules:")
    print("   • Max 50 files changed")
    print("   • Max 1500 lines changed")
    print("   • ONE purpose per PR")
    print("   • Single concern (no mixing API + tests + docs)")
    return 1

</code_context>

<issue_to_address>
**issue (code-quality):** Low code quality found in main - 10% ([`low-code-quality`](https://docs.sourcery.ai/Reference/Default-Rules/comments/low-code-quality/))

<br/><details><summary>Explanation</summary>The quality score for this function is below the quality threshold of 25%.
This score is a combination of the method length, cognitive complexity and working memory.

How can you solve this?

It might be worth refactoring this function to make it shorter and more readable.

- Reduce the function length by extracting pieces of functionality out into
  their own functions. This is the most important thing you can do - ideally a
  function should be less than 10 lines.
- Reduce nesting, perhaps by introducing guard clauses to return early.
- Ensure that variables are tightly scoped, so that code using related concepts
  sits together within the function rather than being scattered.</details>
</issue_to_address>